The invention relates to a memory subsystem and in particular, to a memory subsystem with positional read data latency.
Computer memory subsystems have evolved over the years, but continue to retain many consistent attributes. Computer memory subsystems from the early 1980's, such as the one disclosed in U.S. Pat. No. 4,475,194 to LaVallee et al., of common assignment herewith, included a memory controller, a memory assembly (contemporarily called a basic storage module (BSM) by the inventors) with array devices, buffers, terminators and ancillary timing and control functions, as well as several point-to-point busses to permit each memory assembly to communicate with the memory controller via its own point-to-point address and data bus. FIG. 1 depicts an example of this early 1980 computer memory subsystem with two BSMs, a memory controller, a maintenance console, and point-to-point address and data busses connecting the BSMs and the memory controller.
FIG. 2, from U.S. Pat. No. 5,513,135 to Dell et al., of common assignment herewith, depicts an early synchronous memory module, which includes synchronous dynamic random access memories (DRAMs) 8, buffer devices 12, an optimized pinout, an interconnect and a capacitive decoupling method to facilitate operation. The patent also describes the use of clock re-drive on the module, using such devices as phase lock loops (PLLs).
FIG. 3, from U.S. Pat. No. 6,510,100 to Grundon et al., of common assignment herewith, depicts a simplified diagram and description of a memory system 10 that includes up to four registered dual inline memory modules (DIMMs) 40 on a traditional multi-drop stub bus channel. The subsystem includes a memory controller 20, an external clock buffer 30, registered DIMMs 40, an address bus 50, a control bus 60 and a data bus 70 with terminators 95 on the address bus 50 and data bus 70.
FIG. 4 depicts a 1990's memory subsystem which evolved from the structure in FIG. 1 and includes a memory controller 402, one or more high speed point-to-point channels 404, each connected to a bus-to-bus converter chip 406, and each having a synchronous memory interface 408 that enables connection to one or more registered DIMMs 410. In this implementation, the high speed, point-to-point channel 404 operated at twice the DRAM data rate, allowing the bus-to-bus converter chip 406 to operate one or two registered DIMM memory channels at the full DRAM data rate. Each registered DIMM included a PLL, registers, DRAMs, an electrically erasable programmable read-only memory (EEPROM) and terminators, in addition to other passive components.
As shown in FIG. 5, memory subsystems were often constructed with a memory controller connected either to a single memory module, or to two or more memory modules interconnected on a ‘stub’ bus. FIG. 5 is a simplified example of a multi-drop stub bus memory structure, similar to the one shown in FIG. 3. This structure offers a reasonable tradeoff between cost, performance, reliability and upgrade capability, but has inherent limits on the number of modules that may be attached to the stub bus. The limit on the number of modules that may be attached to the stub bus is directly related to the data rate of the information transferred over the bus. As data rates increase, the number and length of the stubs must be reduced to ensure robust memory operation. Increasing the speed of the bus generally results in a reduction in modules on the bus, with the optimal electrical interface being one in which a single module is directly connected to a single controller, or a point-to-point interface with few, if any, stubs that will result in reflections and impedance discontinuities. As most memory modules are sixty-four or seventy-two bits in data width, this structure also requires a large number of pins to transfer address, command, and data. One hundred and twenty pins are identified in FIG. 5 as being a representative pincount.
FIG. 6, from U.S. Pat. No. 4,723,120 to Petty, of common assignment herewith, is related to the application of a daisy chain structure in a multipoint communication structure that would otherwise require multiple ports, each connected via point-to-point interfaces to separate devices. By adopting a daisy chain structure, the controlling station can be produced with fewer ports (or channels), and each device on the channel can utilize standard upstream and downstream protocols, independent of their location in the daisy chain structure.
FIG. 7 represents a daisy chained memory bus, implemented consistent with the teachings in U.S. Pat. No. 4,723,120. A memory controller 111 is connected to a memory bus 315, which further connects to a module 310a. The information on bus 315 is re-driven by the buffer on module 310a to the next module, 310b, which further re-drives the bus 315 to module positions denoted as 310n. Each module 310a includes a DRAM 311a and a buffer 320a. The bus 315 may be described as having a daisy chain structure with each bus being point-to-point in nature.
One drawback to the use of a daisy chain bus is associated with the return of read data from a series of daisy chained memory modules. As each of the modules in the channel has a different number of intervening stages to return its read data through on its way to the memory controller, modules may have different minimum read data latencies associated with them. Variations in read data return times may be difficult for the memory controller to manage and could result in undue logic complexity. One option for simplifying this situation is to make the read data latency of modules closer to the memory controller in the channel equal to the latency of the module that is furthest away from the memory controller in the channel. Leveling the read data latency in this manner can be achieved by adding a fixed amount of delay to the return of read data based on its location in the daisy chain. In this way, the memory controller will receive all read data with the same latency from the requesting command, regardless of the position within the channel of the addressed module. A major drawback with this approach is that the read data latency is a critical metric of main memory performance. Setting the read data latency of all modules in the daisy chain to be equal to the worst latency results in higher delays incurred, while accessing data from the memory subsystem and therefore decreases system performance.